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ABSTRACT: 

An embodiment of a method and apparatus for facilitating an activity switch between a plurality of 
redundant routing modules (402,404) in a network element (400) is disclosed herein. The 
method includes an operation for receiving a lower layer protocol packet at a lower layer task of a 
first routing module (402). The lower layer protocol packet includes a higher layer protocol packet 
encapsulated therein. An operation is performed for determining a peer network element from 
which the lower layer protocol packet was received. The operation for determining a peer 
network element from which the lower layer protocol packet was received is performed in 
response to failure of a higher layer protocol task (410) of the first routing module (402) while 
processing the higher layer protocol packet In response to determining the peer network 
element from which the lower layer protocol packet was received, an operation is performed for 
terminating higher layer and lower layer protocol sessions of the first routing module (402) with 
the peer network element from which the lower layer protocol packet was received. After 
terminating such higher layer and such lower layer protocol sessions with the peer network 
element from which the lower layer protocol packet was received, an operation is performed for 
implementing an activity switch from the first routing module (402) of the network element to a 
second routing module (404) of the network element (400). 
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(54) Method and apparatus for providing redundant protocol processes in a network element 



(57) An embodiment of a method and apparatus for 
facilitating an activity switch between a plurality of re- 
dundant routing modules (402,404) in a network ele- 
ment (400) is disclosed herein. The method includes an 
operation for receiving a lower layer protocol packet at 
a lower layer task of a first routing module (402). The 
lower layer protocol packet includes a higher layer pro- 
tocol packet encapsulated therein. An operation is per- 
formed for determining a peer network element from 
which the lower layer protocol packet was received. The 
operation for determining a peer network element from 
which the lower layer protocol packet was received is 
performed in response to failure of a higher layer proto- 



col task (410) of the first routing module (402) while 
processing the higher layer protocol packet. In response 
to determining the peer network element from which the 
lower layer protocol packet was received, an operation 
is performed for terminating higher layer and lower layer 
protocol sessions of the first routing module (402) with 
the peer network element from which the lower layer 
protocol packet was received. After terminating such 
higher layer and such lower layer protocol sessions with 
the peer network element from which the lower layer 
protocol packet was received, an operation is performed 
for implementing an activity switch from the first routing 
module (402) of the network element to a second routing 
module (404) of the network element (400). 
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1 EP 1 331 

Description 

FIELD OF THE DISCLOSURE 

[0001] The present invention relates generally to net- 5 
work communications, and more particularly to synchro- 
nization of redundant communication tasks. 

BACKGROUND 

10 

[0002] Data communication protocols serve to facili- 
tate transmission and reception of data across commu- 
nication networks. For example, transmission control 
protocol (TCP), Internet protocol (IP), border gateway 
protocol (BGP), asynchronous transfer mode (ATM), *5 
and various other protocols facilitate communication of 
data between two or more locations in a communication 
network. Through the use of such protocols, communi- 
cation of data across a plurality of communication net- 
works may be facilitated, even though two or more of 20 
the networks comprise different operating systems and 
architectures. 

[0003] The Open Systems Interconnect (OSI) Refer- 
ence Model developed by the International Standards 
Organization (ISO) is generally used to describe the 25 
structure and function of data communications. The OSI 
Reference Model encompasses seven layers, often re- 
ferred to as a stack or protocol stack, which define the 
functions of data communications protocols. The proto- 
col stack comprises a physical layer, a data link layer, a 30 
network layer, a transport layer, a session layer, a pres- 
entation layer, and an application layer. A layer does not 
define a single protocol, but rather a data communica- 
tions function that may be performed by any number of 
protocols suitable to the function of that layer. For ex- 35 
ample, a file transfer protocol and an electronic mail pro- 
tocol provide user services, and are thus part of the ap- 
plication layer. Every protocol communicates with its 
peer, which is a standardized implementation of the 
identical protocol in the equivalent layer on a remote 40 
system. For example, a local electronic mail protocol is 
the peer of a remote electronic mail protocol. As another 
example, BGP on a local router exchanges routing in- 
formation with BGP on a neighboring router. 
[0004] Applications, such as BGP, which require a 
transport protocol to provide reliable data delivery, often 
use TCP because TCP verifies that data is delivered 
across a network (between separate end systems) ac- 
curately and in the proper sequence. TCP provides re- 
liability with a mechanism referred to as Positive Ac- so 
knowledgementwith Retransmission (PAR). In simplest 
terms, a system with PAR re-transmits the data for which 
it has not received an acknowledgement message from 
a far- end node. Information is communicated between 
cooperating TCP modules in segments. A segment is a 55 
datagram containing a TCP header and perhaps data. 
The TCP header contains sequence numbers. Control 
information, called a handshake, is exchanged between 
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the two endpoints to establish a dialogue before data is 
transmitted. 

[0005] As previously discussed, border gateway pro- 
tocol (BGP) typically runs over TCP (e.g., port 179). 
BGP version 4 (BGP4) is the current de facto exterior 
routing protocol for inter-domain (autonomous systems) 
routing. BGP is a protocol used to advertise routes be- 
tween networks of routers, e.g., between a Service Pro- 
vider's network and a Carrier's network. Routers at the 
edges of these networks exchange BGP messages, 
which could affect hundreds of thousands of routes. If 
the BGP process at one of these edge routers termi- 
nates (e.g., because of a restart, hardware failure, soft- 
ware upgrade, etc.), service on the routes between the 
networks is usually affected. The termination also caus- 
es additional BGP messages to be exchanged between 
other edge routers to update information about available 
routes. Consequently, the termination results in a period 
of route instability and unavailability of the affected rout- 
er, which consequences are desirable to avoid. Further- 
more, the termination will often result in a flood of re- 
routing messages being sent into the network, thus ad- 
versely affecting performance of the network. 
[0006] A conventional BGP redundancy technique for 
addressing BGP process failures involves configuring 
two or more routers from different vendors in parallel. 
The objective of such a technique is to reduce the po- 
tential for BGP process failures by relying on the as- 
sumption that one of the routers will survive at least 
some of the time a particular set of circumstances that 
might lead to failure of another router. For example, at 
least one of the routers would ideally exhibit immunity 
to failure such as those that might be caused by an of- 
fending message, a hardware fault, or a software fault. 
That is, it is assumed that routers from different vendors 
are susceptible to different types of failures. This type 
of conventional BGP redundancy technique is generally 
expensive due to the inherent cost of the multiple routers 
and because using equipment from multiple vendors 
causes additional operation, support, network manage- 
ment, and training costs. Additionally, this type of con- 
ventional BGP redundancy technique requires addition- 
al BGP messages to be exchanged to move the routes 
onto the tandem router, thus increasing cost, complex- 
ity, and network traffic. The attached routers still notice 
that the first router has disappeared and then route 
around it. Accordingly, it is desirable to avoid the disad- 
vantages associated with such a conventional BGP re- 
dundancy technique. 

[0007] A graceful restart mechanism for a router is an- 
other conventional technique for addressing BGP proc- 
ess failures. Such a graceful restart mechanism is pro- 
posed in an Internet Engineering Task Force (IETF) draft 
entitled "Graceful Restart Mechanism for BGP". In this 
proposal, a router has the capability of preserving its for- 
warding state (routes) over a BGP restart, the ability to 
notify its peer routers of this capability and the ability to 
notify its peer routers of an estimated time for restart 
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completion before it initiates such a restart. Upon de- 
tecting that the BGP process of the router has terminat- 
ed (i.e., a failed router) and in response to receiving a 
corresponding notification, the peer routers do not send 
new best routes to accommodate for the failed router 
unless It fails to restart within the specified time limit. 
[0008] Such a graceful restart mechanism requires 
that the peer routers be able to interpret and respond to 
the restart notification. Additionally, while the failed rout- 
er is restarting it cannot process routing updates that 
would normally be received. Consequently, it becomes 
out of date during the period of unavailability, which is 
followed by a burst of updates once back in service 
These updates cause increased "chum" in the routing 
tables of other routers, which affects performance of the 
network and should therefore be avoided. Even worse, 
routing loops or "blackholes" may form in this period of 
unavailability. Such "blackholes" occur when a route is 
advertised as available, but when the corresponding 
router is not actually configured to support such a route, 
resulting in loss of packets intended to be communicat- 
ed over that route. Furthermore, the router may not ac- 
tually be coming back into service. Also, since a graceful 
restart mechanism allows the specified time limit for 
routers to be restarted, waiting that amount of time can 
increase the time it takes to detect a failure and route 
around the failed router. Additionally, implementation of 
such a grateful restart mechanism requires protocol ex- 
tensions to BGP to which all routers aware of the failure 
must adhere in order to support the graceful restart 
mechanism. Accordingly, it is desirable to avoid the dis- 
advantages associated with a graceful restart mecha- 
nism. 

[0009] Therefore, facilitating synchronization of pro- 
tocol tasks and related information on redundant routing 
modules of a network element in a manner that enables 
limitations associated with conventional redundancy 
techniques to be overcome is useful. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0010] 

FIG. 1 is flow chart view depicting a method for syn- 
chronizing TCP tasks running on redundant routing 
modules of a network element in accordance with 
an embodiment of the disclosures made herein. 
FIG. 2 is a flow chart view depicting a method for 
facilitating an activity switch in accordance with an 
embodiment of the disclosures made herein. 
FIG. 3 is a flow chart view depicting a method for 
synchronizing routing protocol information associ- 
ated with a plurality of routing modules of a network 
element in accordance with an embodiment of the 
disclosures herein. 

FIG. 4 is a block diagram view depicting a network 
element 400 capable of carrying out methods in ac- 
cordance with embodiments of the disclosures 



made herein. 
DETAILED DESCRIPTION OF THE FIGURES 

5 [001 1 ] An embodiment of a method and apparatus for 
facilitating an activity switch between a plurality of re- 
dundant routing modules in a network element is dis- 
closed herein. The method includes an operation for re- 
ceiving a lower layer protocol packet at a lower layer 

10 task of a first routing module. The lower layer protocol 
packet includes a higher layer protocol packet encap- 
sulated therein. An operation is performed for determin- 
ing the peer network element from which the higher lay- 
er protocol packet was received. The operation for de- 

15 termining a peer network element from which the higher 
layer protocol packet was received occurs for each re- 
ceived packet. Identification of the peer network ele- 
ment is stored for each received packet. As such, if a 
failure occurs, the identification of the peer network el- 

20 ement can be retrieved and appropriate action taken. In 
response to determining the occurrence of a failure, an 
operation is performed for terminating higher layer and 
lower layer protocol sessions of a second routing mod- 
ule of the network element with the peer network ele- 

25 ment from which the higher layer protocol packet was 
received. After terminating such higher layer and such 
lower layer protocol sessions with the peer network el- 
ement from which the higher layer protocol packet was 
received, an operation is performed for implementing an 

30 activity switch from the first routing module of the net- 
work element to the second routing module of the net- 
work element. 

[001 2] The disclosures made herein pertain to various 
aspects of facilitating synchronization of redundant rout- 
35 jng modules in a network element. In accordance with 
embodiments of the disclosures made herein, lower lay- 
er protocol (e.g., Transmission Control Protocol (TCP)) 
and higher layer protocol (e.g., Border Gateway Proto- 
col (BGP)) tasks of a first routing module are synchro- 
40 nized with respective lower layer protocol (e.g., TCP) 
and higher layer protocol (e.g., BGP) tasks of a second 
routing module. The first routing module and the second 
routing module are redundant routing modules within a 
network element. Protocol information (e.g., TCP pack- 
45 ets, BGP packets, etc) that is processed on the first rout- 
ing module (e.g., an active one of a plurality of redundant 
routing modules) is similarly processed on the second 
routing module (e.g., an inactive one of the plurality of 
redundant routing modules). Accordingly, such a net- 
so work element in accordance with an embodiment of the 
disclosures made herein advantageously comprises re- 
dundant, synchronized routing modules that are capa- 
ble of supporting carrier-grade quality of service over 
networks of various communication protocols (e.g., In- 
55 ternet Protocol, etc). A lower layer protocol (e.g., TCP) 
packet (which may be referred to as a segment) is not 
necessarily congruent with a higher layer protocol (e.g., 
BGP) packet. For example, it is not necessarily true that 
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a TCP packet contains a BGP packet. For example, say 
a node is transmitting two BGP packets A and B and 
each packet includes 1000 bytes. A TCP task will most 
likely transmit portions of these BGP packets in sepa- 
rate TCP packets. For example, a first TCP segment 
may contain 5 1 2 bytes of data (the first 5 1 2 bytes of BG P 
packet A), a second TCP segment may contain 512 
bytes of data (the remaining 488 bytes of BGP packet 
A, together with the first 24 bytes of BGP packet B), a 
third TCP segment may contain 512 bytes of data (the 
next 512 bytes of BGP packet B), and finally a fourth 
TCP segment may contain 464 bytes of data (the re- 
maining 464 bytes of BGP packet B). The foregoing is 
merely an example, and other relationships between 
lower layer protocol packets and higher layer protocol 
packets are entirely possible. 

[0013] Embodiments of the disclosures made herein 
are capable of enabling redundant lower layer protocol 
tasks (e.g., TCP tasks) and higher layer protocol tasks 
(e.g., BGP tasks), thus allowing for an activity switch 
without adversely affecting service. In such embodi- 
ments, when an activity switch is implemented, disrup- 
tion of service on routes distributed by such higher layer 
and lower layer protocols is limited, if not eliminated. For 
example, after such an activity switch, a newly active 
routing module (i.e., previously the inactive routing mod- 
ule) processes routing updates that would normally be 
received by a newly inactive routing module (i.e., previ- 
ously the active routing module). Furthermore, the new- 
ly active routing module does not become out of date 
with respect to routing information maintained on other 
network elements. In this manner, the network is not bur- 
dened by a burst of updates in response to the activity 
switch. Limiting the burden of such a burst of updates 
eliminates "churn" in the routing tables of network ele- 
ments, thus improving performance of the network. Sig- 
nificantly, service of existing routes is maintained, and 
change to existing routes, deletion of routes, and addi- 
tion of routes can continue uninterrupted; the switchover 
is transparent to neighboring routers. By being transpar- 
ent to neighboring routers, a technique disclosed herein 
need not require cooperation of neighboring routers to 
enable an activity switch. Accordingly, neighboring rout- 
ers need not be made aware of such an activity switch, 
nor do they need to support protocol extensions to en- 
able such an activity switch. 

[0014] Such embodiments are advantageous in that 
an offending packet of information that results in failure 
of a higher layer protocol task of a first routing module 
does not readily result in failure of the same higher layer 
protocol task of a second routing module that is redun- 
dant and synchronized with respect to the first routing 
module. One embodiment of a technique for limiting the 
potential for failure of the second routing module from 
the offending packet of information is to maintain a high- 
er layer protocol task (e.g., a BGP task) of an inactive 
one of a plurality of synchronized routing modules (i.e., 
an inactive routing module) at least one higher layer pro- 



tocol packet (e.g., a BGP packet) behind the same high- 
er layer protocol task of an active one of the plurality of 
synchronized routing modules (i.e., the active routing 
module). In this manner, the offending packet is recog- 
5 nized as such prior to being processed by the higher 
layer protocol task of the inactive routing module, there- 
by avoiding the failure of the higher layer protocol task 
of the inactive routing module that would otherwise re- 
sult. 

10 [0015] Another advantageous aspect of embodi- 
ments of the disclosures made herein is that processing 
power of a network element is not adversely affected. 
Specifically, synchronization and redundancy in accord- 
ance with embodiments of the disclosures made herein 

is are facilitated in an efficient and effective manner. Ac- 
cordingly, a significant majority of processing power of 
the network element is available for performing primary 
tasks of the network element (e.g., switching, routing, 
etc). 

20 [001 6] Still another advantageous aspect of embodi- 
ments of the disclosures made herein is that such em- 
bodiments are less costly to implement and maintain 
than conventional solutions. Such embodiments do not 
require redundant network elements, but rather redun- 

25 dant routing modules within a particular network ele- 
ment. In some embodiments, the redundant routing 
modules are implemented identically, thus reducing 
cost. For example, similar software may be executed 
within each of the redundant routing modules. In other 

30 embodiments, differently-implemented redundant rout- 
ing modules may be used. 

[0017] ft should be understood that embodiments of 
the present invention may be practiced with a variety of 
higher layer protocols. While BGP packets are men- 

35 tioned in many places herein, ft should be understood 
that routes can also arrive from other protocols (e.g., 
Open Shortest Path First (OSPF)), or due to configura- 
tion changes (e.g., static routes). Not only are routes 
kept is sync between an active routing module and an 

^0 inactive routing module, but so is configuration. The 
configuration can also change on-the-fly (e.g., a BGP 
peer may be added or removed at any time). It should 
be noted that a higher layer protocol may be used for 
advertising routes, but may also be used for withdrawing 

45 routes (e.g., a BGP packet may also specify a route to 
withdraw). 

[0018] Turning now to the figures, a method 100 for 
synchronizing TCP tasks running on redundant routing 
modules of a network element in accordance with an 

50 embodiment of the disclosures made herein is depicted 
in FIG. 1 . The method is performed by a network ele- 
ment preferably comprising a line card 134, an active 
routing module 1 36, and an inactive routing module 1 38 . 
Various steps of the method are illustrated as being per- 

55 formed by the line card 1 34, by the active routing module 
1 36, and by the inactive routing module 1 38. The meth- 
od 100 begins at an operation 102 where a line card of 
a network element forwards a Protocol Data Unit (PDU), 
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or a copy thereof , for reception by an active routing mod- 
ule and an inactive routing module of the network ele- 
ment. An Internet Protocol routing module is an example 
of both the active and the inactive routing modules. An 
apparatus capable of providing routing functionality (i. 
e. t a router) is an example of the network element. In 
other embodiments, the network element need not be 
implemented on a router, but may by implemented on 
one or more other network devices. As an example, for 
embodiments wherein the higher layer protocol packets 
are Multi-Protocol Label Switching (MPLS) packets, the 
network element may be so implemented. Accordingly, 
the active routing module and inactive routing module 
may be considered more generically to be simply an ac- 
tive module and an inactive module. The active routing 
module performs an operation 104 for receiving the 
PDU while the inactive routing module effectively ig- 
nores (e.g., receives but not process) the PDU in oper- 
ation 140. 

[0019] After receiving the PDU, the active routing 
module performs an operation 1 06 for extracting a TCP 
packet encapsulated within the PDU. The TCP packet 
extracted from the PDU is hereinafter referred to as the 
inbound TCP packet. The active routing module TCP 
task performs an operation 110 for receiving the first 
copy of the inbound TCP packet. 
[0020] After the active routing module receives the 
first copy of the inbound TCP packet, the active routing 
module TCP task performs an operation 1 1 4 for storing 
the first copy of the inbound TCP packet in a receive 
queue associated with the active routing module TCP 
task. After operation 114, the active routing module per- 
forms operation 142 to make a determination as to 
whether or not the inbound TCP packet should be for- 
warded to the inactive routing module 138. If it is deter- 
mined that the inbound TCP packet should not be for- 
warded, the process continues to operation 144, where 
the inbound TCP packet is not forwarded. If it is deter- 
mined that the inbound TCP packet should be forward- 
ed, the process continues to operations 1 08 and 120. In 
operation 1 08, the active routing module forwards a first 
copy of the inbound TCP packet for reception by a TCP 
task of the active routing module (i.e., the active routing 
module TCP task) and a second copy of the inbound 
TCP packet for reception by a TCP task of the inactive 
routing module (i.e., the inactive routing module TCP 
task). In at least one embodiment, operation 114 is per- 
formed before operation 1 08, while, in at least one em- 
bodiment, operation 1 08 is performed before operation 
1 1 4. It is important to note that the active routing module 
processes the incoming TCP packet and then, if appro- 
priate, forwards the incoming TCP packet (along with 
other information) to the inactive routing module. Some 
incoming TCP packets, for example, acknowledge- 
ments that contain no data, need not be forwarded to 
the inactive routing module. The inactive routing module 
TCP task performs an operation 112 for receiving the 
second copy of the inbound TCP packet. Similarly, after 



the inactive routing module receives the second copy of 
the inbound TCP packet, the inactive routing module 
TCP task performs an operation 1 1 6 for storing the sec- 
ond copy of the inbound TCP packet in a receive queue 
5 associated with the inactive routing module TCP task. 
The operation 1 1 6 for storing the second copy of the in- 
bound TCP packet in a receive queue associated with 
the inactive routing module TCP task includes initially 
storing second copy of the inbound TCP packet in a 
10 pending portion of the receive queue associated with the 
inactive routing module TCP task (i.e., the pending por- 
tion of the inactive routing module receive queue). 
[0021 ] In operation 1 20, a BGP task of the active rout- 
ing module (i.e., the active routing module BGP task) 

is facilitates recordation of the peer network element from 
which the inbound TCP packet was received. As dis- 
cussed below in greater detail in reference to FIG. 2, 
such a record of the peer network element from which 
the inbound TCP packet was received is used for facil- 

20 itating an activity switch from the active routing module 
to the inactive routing module if a failure occurs while 
processing the BGP packet. After operation 1 20, the ac- 
tive routing module BGP task performs an operation 118 
for processing the BGP message. 

25 [0022] After operation 118, an operation 121 is per- 
formed for determining whether processing of the BGP 
message contained in the first copy of the inbound TCP 
packet is performed successfully. When the operation 
1 1 8 for processing a BGP message contained in the first 

30 copy of the inbound TCP packet is successfully per- 
formed, the inactive routing module TCP task performs 
an operation 122 for storing the second copy of the in- 
bound TCP packet in a ready portion of the receive 
queue associated with the inactive routing module TCP 

35 task (i.e;, the ready portion of the inactive module re- 
ceive queue). In at least one embodiment of the opera- 
tion 122, the operation 122 for storing the second copy 
of the inbound TCP packet in the ready portion of the 
inactive routing module receive queue includes forward- 

io ing the second copy of the inbound TCP packet from the 
pending portion to the ready portion of the inactive rout- 
ing module receive queue. Upon a determination in op- 
eration 121 that processing of the BGP message con- 
tained in the first copy of the inbound TCP packet was 

45 performed successfully, the second copy of the inbound 
TCP packet can be immediately moved from the pend- 
ing portion to the ready portion of the inactive routing 
module receive queue, but, in at least one embodiment, 
it is advantageous to cause such action to occur at a 

50 later time for performance reasons. For example, an in- 
struction for the inactive routing module to perform op- 
eration 122 may be included within other information 
destined for the inactive routing module to avoid the 
need to send the instruction separately and to minimize 

55 the amount of information being sent to the inactive rout- 
ing module and the amount of processing required by 
the inactive routing module. 

[0023] After, and only after, the second copy of the 
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inbound TCP packet is stored in the ready portion of the 
inactive routing module receive queue, the inactive rout- 
ing module BGP task performs an operation 124 for 
processing the BGP message contained in the second 
copy of the inbound TCP packet. The active routing 
module performs an operation 126 for issuing an ac- 
knowledgement message for designating that the in- 
bound TCP packet has been received. In at least one 
embodiment, operation 126 is performed after operation 
122, while in at least one other embodiment, operation 
126 is performed before operation 122, as long as it is 
performed after operation 116. The operation for initially 
storing the second copy of the inbound TCP packet in 
the pending portion of the inactive routing module re- 
ceive queue and then in the ready portion of the inactive 
routing module receive queue enables the second copy 
of the inbound TCP packet to remain unprocessed by 
the inactive routing module BGP task until the contents 
of the inbound TCP packet is determined to be non- of- 
fensive (e.g., not causing a BGP task failure) via the ac- 
tive routing module BGP task successfully processing 
the first copy of the inbound TCP packet. In this manner, 
the inactive routing module BGP task processes a par- 
ticular BGP messages after the active routing module 
BGP task processes the particular BGP message. 
[0024] In at least one embodiment of the active rout- 
ing module and inactive routing module BGP tasks, 
such BGP tasks are precluded from receiving partial 
TCP packets from the TCP task. Such partial packets 
may contain partial BGP messages therein, potentially 
causing synchronization problems when an activity 
switch is implemented. It is contemplated herein that a 
TCP task may be configured such it precludes an asso- 
ciated BGP task from recognizing that information of a 
packet is being received until the information comprises 
a full packet- A socket option exists for enabling such 
functionality. 

[0025] By enabling the BGP message contained in 
the second copy of the inbound TCP packet to be proc- 
essed only after second copy of the TCP packet is 
stored in the ready portion of the inactive routing module 
receive queue, it is assured that such processing of the 
BGP message contained in the second copy of the TCP 
packet will take place only after the BGP message con- 
tained in the first copy of the TCP packet is successfully 
processed by the active routing module BGP task. It 
should be understood that the BGP messages con- 
tained in the first copy and the second copy of the in- 
bound TCP packet are essentially identical (i.e., the 
same BGP message). Accordingly, the potential for fail- 
ure of the active and inactive routing module BGP tasks 
resulting from the same BGP message is substantially 
reduced, if not eliminated. 

[0026] Issuing the acknowledgement message for 
designating that the inbound TCP packet has been re- 
ceived (e.g., operation 126) only after the second copy 
of the TCP packet has been stored in the inactive routing 
module receive queue pending portion (e.g., operation 



1 1 6) assures that the inactive routing module will not fail 
to receive any TCP packets, even in the event of an ac- 
tivity switch. Thus, in he event that an activity switch 
does not occur, the BGP message contained in the first 

5 copy of the TCP packet is successfully processed by the 
active routing module BGP task. Moreover, upon an ac- 
tivity switch, the TCP communication with the peer net- 
work element that transmitted a TCP packet including 
an offending BGP message is terminated. This opera- 

io tional sequence ensures that a TCP packet including an 
offending BGP message is not processed after such 
message results in failure of the active routing module 
BGP task. Thus, redundancy robustness is enhanced. 
[0027] Turning now to update messages being trans- 

* 5 mitted from the network element for reception by its peer 
network elements, it will be appreciated that redundancy 
can also be provided where such outbound update mes- 
sages are concerned. For example, in response to re- 
ceiving a BGP message designating a new route, a 

20 route update message is transmitted from the network 
element for reception by one or more of its peer network 
elements for notifying such peer network elements of 
the new route. 

[0028] Accordingly, in response to the active routing 

25 module receiving such types of BGP messages that ne- 
cessitate an outbound update message or receiving a 
route from another protocol (e.g., OSPF or ISIS), or in 
response to an internal event (e.g., a configuration 
change, such as, for example, adding a static route) for 

30 which an update message should be generated, an op- 
eration 128, FIG. 1 , is performed for storing a first copy 
of an outbound BGP packet encapsulated within one or 
more TCP packets in a transmit queue of the active rout- 
ing module (i.e., the active routing module transmit 

35 queue). Operation 1 28 occurs after operation 1 1 8. Also, 
in response to the active routing module receiving such 
types of BGP messages that involve an outbound up- 
date message, an operation 130 is performed for storing 
a second copy of the outbound BGP packet encapsu- 

40 lated within one or more TCP packets in a transmit 
queue of the inactive routing module (i.e., the inactive 
routing module transmit queue). Similar to receive 
queue functionality as disclosed herein, in at least one 
embodiment of the active routing module and inactive 

45 routing module transmit queues, such transmit queues 
are precluded from storing partial BGP packets. An op- 
eration 132 is performed for forwarding the first copy of 
the outbound BGP packet encapsulated within one or 
more TCP packets from the active routing module trans- 

50 mit queue for reception by one or more peer network 
elements only after the second copy of the outbound 
BGP packet encapsulated within one or more TCP 
packets is stored in the inactive routing module transmit 
queue. In this manner, retransmission and packet se- 

55 quencing functionality are maintained after an activity 
switch from the active routing module to the inactive 
routing module. 

[0029] Referring back to the operation 1 21 , this oper- 
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ation is also capable of determining whether processing 
of the BGP message contained in the first copy of the 
inbound TCP packet is not successfully performed. In 
response to the BGP message contained in the first 
copy of the inbound TCP packet being not being proc- 
essed successfully by the active routing module BGP 
task, an activity switch is facilitated, and the process is 
directed to an entry point "A." The activity switch trans- 
fers on-line operations of the network element from the 
previously active routing module (now the inactive rout- 
ing module) to a newly active routing module (previously 
the inactive routing module). 

[0030] FIG. 2 depicts a method 200 for facilitating an 
activity switch in accordance with an embodiment of the 
disclosures made herein. The method 200 pertains to 
an activity switch resulting from an offending inbound 
TCP packet. A TCP packet including an offending BGP 
message is one example of the offending TCP packet. 
[0031 ] At an entry point "A" corresponding to process- 
ing of an errant BGP packet of an inbound TCP packet 
(i.e., one for which satisfactory error handling has not 
otherwise been provided), the method 200 begins with 
an error handling routine invoked by a system controller 
implements an operation 202 for identifying a peer net- 
work element from which the offending inbound BGP 
packet was received. One embodiment of identifying the 
identified peer network element includes reading/ac- 
cessing a record generated in response to the operation 
120, FIG. 1, for facilitating recordation of the peer net- 
work from which the offending inbound TCP packet was 
received. It should be noted that, in at least one embod- 
iment, when recording the network element from which 
a packet has been received, the peer network element 
of the BGP packet is recorded, not that of the TCP seg- 
ment. It is possible that the BGP peer network element 
and TCP peer network element may be different. BGP 
may have a session with a neighbor that requires mul- 
tiple TCP hops to reach. As such, the identified peer net- 
work element is identified with respect to the higher layer 
protocol packet (e.g., the BGP packet). 
[0032] The system controller may, for example, be a 
control element, such as a processor, coupled to the net- 
work element or incorporated within the network ele- 
ment. For example, the system controller may be imple- 
mented as a process that reads the record to determine 
the peer network element from which the packet was 
received and to initiate the termination of the associated 
BGP session on the inactive routing module. In at least 
one embodiment, this process is contained within the 
active routing module. When required to terminate a 
session, the active routing module (e.g., the system con- 
troller contained within the active routing module) com- 
municates with the inactive routing module as to which 
peering session to terminate. 

[0033] After identifying the peer network element from 
which the offending inbound BGP packet was received 
(i.e., the identified peer network element), the error han- 
dling routing performs an operation 204 for initiating ter- 



mination of a BGP session associated with the identified 
peer network element and an operation 206 for initiating 
termination of a TCP session associated with the iden- 
tified peer network element. Since, in at least one em- 
5 bodiment, initiating termination of a BGP session will in- 
herently initiate termination of a TCP session, opera- 
tions 204 and 206 may optionally be performed as a sin- 
gle operation. Likewise, such a single operation may re- 
sult in performance of operation 208, which may inher- 

10 entry result in performance of operation 21 0. After initi- 
ating termination of the BGP and TCP sessions associ- 
ated with the identified peer network element, the newly 
active routing module TCP task performs an operation 
21 0 for terminating the TCP session associated with the 

15 identified peer network element, and the soon-to-be- 
newly active routing module BGP performs an operation 
208 for terminating the BGP session associated with the 
identified peer network element. In response to facilitat- 
ing termination of the TCP session associated with the 

20 identified peer network element, the newly active routing 
module TCP task performs an operation 21 2 for purging 
the offending inbound TCP packet from the receive 
queue of the newly active routing module. The actual 
switching of functional operations is facilitated after the 

25 TCP and BGP sessions are terminated and the offend- 
ing inbound TCP packet is purged from the receive 
queue of the newly active routing module. Because the 
TCP session has been terminated, even if the offending 
inbound TCP packet has not been acknowledged, it will 

30 not be re-sent, thereby avoiding a failure of the newly 
active routing module. 

[0034] As an additional precaution, TCP and BGP 
task sessions with the identified peer network element 
are re-established after an operation 214 is performed 

35 for restarting the newly inactive module and until after 
an operation 21 6 is performed for synchronizing existing 
routing-related information of the newly inactive routing 
module with the newly active routing module. Such rout- 
ing-related information may include information stored 

40 j n a routing information database, as well as other infor- 
mation, such as configuration information (e.g., static 
configuration information) and state information (e.g., 
dynamic state information). In response to synchroniz- 
ing such existing routing-related information, the newly 

45 active routing module BGP task performs an operation 
220 for re-establishing a BGP session with the identified 
peer network element. To re-establish a BGP session in 
accordance with operation 220, the newly active routing 
module TCP task performs an operation 218 for re-es- 

50 tablishing a TCP session with the identified peer net- 
work element. In this manner, risk associated with re- 
establishing such task sessions with the identified net- 
work element without a redundant routing module being 
in place are reduced, if not eliminated. Optionally, in at 

55 least one embodiment, BGP and TCP task sessions are 
maintained with other peer network elements besides 
the identified peer network element. 
[0035] FIG. 3 depicts a method 300 for synchronizing 
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routing protocol information associated with a plurality 
of routing modules of a network element in accordance 
with an embodiment of the disclosures herein. By syn- 
chronizing such routing protocol information, redundan- 
cy in accordance with the disclosures made herein may 
be implemented. Such synchronization contributes to 
enabling an activity switch from a first routing module of 
the network element to a second routing module of the 
network element in an essentially transparent manner 
with respect to peer network elements. 
[0036] The method 300 begins with an inactive rout- 
ing module performing an operation 302 for receiving a 
copy of existing routing protocol information from an ac- 
tive routing module. The operation 302 is performed in 
response to the inactive routing module being an addi- 
tional routing module that is added to a network element 
including the active routing module. Because the active 
routing module is an existing, in-use routing module of 
the network element, the active routing module has such 
existing routing protocol information associated there- 
with prior to addition of the inactive routing module. For 
example, it may be the case that at least a portion of the 
routing protocol information was dynamically estab- 
lished in the existing routing module over a period of 
time prior to the addition of the additional routing module 
to the network element. Examples of routing protocol in- 
formation include TCP related state information, BGP 
configuration, BGP routing tables, and route state infor- 
mation (e.g., designation that a route has been adver- 
tised to peer network elements). 
[0037] In response to the inactive routing module re- 
ceiving such existing routing information, an operation 
304 is performed for updating inactive routing module 
records associated with such routing protocol informa- 
tion. An embodiment for updating such inactive routing 
module records associated with such existing routing 
protocol information includes updating a routing infor- 
mation database of the inactive routing module. In one 
embodiment of the inactive routing module, the inactive 
routing module does not include any existing routing 
protocol information (e.g., the inactive routing module is 
a new routing module being put into service). In another 
embodiment of the inactive routing module, the inactive 
routing module includes existing routing protocol infor- 
mation that is being updated. 

[0038] At some point in time after the inactive routing 
module is added to the network element and during the 
normal course of operation of the active routing module, 
the active routing module performs an operation 306 for 
receiving a first copy of new routing protocol information 
(newly- received routing protocol information) from one 
ormore peer network elements. In response to receiving 
such newly-received routing protocol information, the 
active routing module performs an operation 308 for up- 
dating active routing module records associated with 
such newly-received routing protocol information, an 
operation 312 for forwarding a second copy of such 
newly-received routing protocol information for recep- 



tion by the inactive routing module, and an operation 
310 for acknowledging receipt of such newly received 
routing protocol information. Thus, acknowledgement is 
provided to the one or more peer network elements from 
5 which the new routing protocol information was received 
after a copy of such new routing protocol information (or 
the portion thereof for which the acknowledgement is 
being provided) has been forwarded to the inactive rout- 
ing module (i.e., the additional routing module). After the 
io active routing module forwards such newly-received 
routing protocol information for reception by the inactive 
routing module, the inactive routing protocol performs 
an operation 31 4 for receiving such newly-received rout- 
ing protocol information and an operation 31 6 for updat- 
es jng inactive routing module records associated with 
such routing protocol information. It should be noted that 
operations 304 and 316 may be performed as separate 
operations or combined into a single operation. A TCP 
packet including a BGP message is an example of such 
20 newly-received routing protocol information during the 
normal course of operation ofthe active routing module. 
[0039] Referring now to FIG. 4, a network element 
400 capable of carrying out methods in accordance with 
embodiments of the disclosures made herein is deplet- 
es ed. Specifically, the network element 400 is capable of 
carrying out redundancy and synchronization function- 
ality in accordance with the disclosures made herein. 
For example, the network element 400 is capable of car- 
rying out the methods disclosed herein (e.g., the meth- 
30 ods 1 00, 200 and 300). An apparatus capable of provid- 
ing routing functionality (e.g., a router) is an example of 
the network element 400. 

[0040] The network element 400 includes an active 
routing module 402 (i.e., the first routing module), an 

35 inactive routing module 404 (i.e., the second routing 
module), and a line card 405 connected between the 
active and inactive routing modules (402, 404). The line 
card facilitates routing a respective copy of each in- 
bound TCP packet (e.g., via forwarding of correspond- 

40 ing Protocol Data Units (PDUs)). However, the TCP task 
of the inactive routing module 402 ignores such TCP 
packets (e.g., does not process the PDUs) while the 
TCP task of the active routing module 402 processes 
such TCP packets. 

45 [0041] The active routing module 402 and the inactive 
routing module 404 are capable of facilitating redundant 
functionality in according with the disclosures made 
herein. The active routing module 402 and the inactive 
routing module 404 each include respective TCP tasks 

50 (406, 408), respective BGP tasks (410, 412) and re- 
spective routing information databases (414, 416). The 
TCP tasks (406, 408) are each examples of lower layer 
protocol tasks. The BGP tasks (410, 412) are each ex- 
amples of higher layer protocol tasks. It is contemplated 

55 herein that BGP tasks (410, 412) may be substituted 
with other protocols that use TCP to exchange messag- 
es (e.g., multi-protocol label switching (MPLS)). 
[0042] The TCP task 406 of the active routing module 
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includes a receive queue 41 8 and a transmit queue 420. 
The TCP task 408 of the inactive routing module 404 
includes a receive queue 422 and a transmit queue 424. 
The receive queue 422 includes a pending portion 426 
and a ready portion 428. The pending portion 426 and 
the ready portion 428 of the inactive routing module re- 
ceive queue 422 facilitate functionality as depicted in 
FIG. 1 . Specifically, the pending portion 426 and the 
ready portion 428 of the inactive routing module receive 
queue 422 enables a particular copy of a TCP packet to 
remain unprocessed by the inactive routing module 
BGP task 412 until the contents of such TCP packet is 
determined to be non-offensive (e.g., not causing a BGP 
task failure) by the BGP task 410 of the active routing 
module 402. This, the inactive routing module BGP task 
412 processes a particular BGP message after the ac- 
tive routing module BGP task 410 processes the partic- 
ular BGP message. 

[0043] In some embodiments of the inactive routing 
module, the inactive routing module 404 does not re- 
ceive flow control updates from the active routing mod- 
ule 402. Thus, it is theoretically possible for the inactive 
routing module receive queue 422 to overflow. To re- 
duce this possibility, the inactive routing module receive 
queue 422 is preferably much larger than the active rout- 
ing module receive queue 418 in such embodiments. 
However, it should be understood that the inactive rout- 
ing module 404 does less work than the active routing 
module 402 (e.g., the flooding responsibilities are great- 
ly reduced, the TCP/IP stack is not transmitting data, 
etc.). Accordingly, there should not be a steady state 
possibility where the inactive routing module receive 
queue 422 continues to grow without limit. 
[0044] It should be understood that the active routing 
module 402 is capable of supporting functionality dis- 
closed herein in association with the inactive routing 
module 404 and the inactive routing module 404 is ca- 
pable of supporting functionality disclosed herein in as- 
sociation with the active routing module 402. According- 
ly, in the event of an activity switch in accordance with 
the disclosures made herein, the active routing module 
402 (i.e., the newly inactive routing module) provides 
functionality previously provided by the inactive routing 
module 404 (i.e., the newly active routing module) and 
the inactive routing module 404 provides functionality 
previously provided by the active routing module 402. 
For example, after an activity switch, the active routing 
module 402 provides functionality associated with the 
pending queue 426 and ready queue 428 of the inactive 
routing module 404. 

[0045] In accordance with at least one embodiment of 
the disclosures made herein, the BGPtasks of the active 
routing module 402 and the inactive routing module 404 
do not queue any transmit data on a per-peer (i.e., per 
socket) basis. One reason that the BGP tasks no longer 
queue on a per-peer basis is because data queued in 
the BGP task would not be guaranteed of delivery after 
an activity switch. Another reason is that synchroniza- 



tion of lists of routes which need to be advertised or with- 
drawn would be excessively intensive if BGP task trans- 
mit queues needed searching. 
[0046] It is contemplated herein that the active routing 

s module transmit queue 420 is enlarged in order to ena- 
ble omission of a transmit queue of the active routing 
module BGP task. That is, the transmit queue 420 of the 
active routing module TCP task 406 needs to be large 
enough to ensure that transmissions continue between 

10 successive periods of processing of advertised or with- 
drawn routes. 

[0047] Because the BGP tasks (41 0, 412) of the ac- 
tive and inactive routing modules (402, 404) cannot 
queue any transmit data, an operation for transmitting 

15 data to the active routing module TCP task 406 must 
succeed. Otherwise, the active routing module BGP 
task 41 0 would have to queue such transmit data, which 
it preferably does not do. To ensure that the operation 
for transmitting data to the active routing module TCP 

20 task 406 succeeds, the active routing module BGP task 
41 0 first ensures that sufficient space exists in the trans- 
mit queue 420 associated with the active routing module 
TCP task 406. In one embodiment, ensuring that such 
sufficient space exists is accomplished via a read in 

25 shared memory. To this end, the TCP task 406 of the 
active routing module 402 maintains a table of free 
space in the active routing module transmit queue 420. 
However, in other embodiments, other techniques may 
be used for ensuring that such sufficient space exists. 

30 [0048] Referring now to data processor programs in 
accordance with an embodiment of the disclosures 
made herein, a data processor program controls at least 
a portion of the operations associated with synchroniz- 
ing higher layer protocol tasks (e.g. , BGP) and lower lay- 

35 er protocol tasks (e.g., TCP) running on redundant rout- 
ing modules of a network element. In this manner, the 
data processor program controls at least a portion of the 
operations necessary to facilitate routing module syn- 
chronization functionality in a manner consistent with 

40 the disclosures made herein. The term data processor 
program is defined herein to refer to computer software, 
data processor algorithms or any other type of instruc- 
tion code capable of controlling operations associated 
with a data processor. A microprocessor, microcontrol- 

45 |er, microcomputer, digital signal processor, state ma- 
chine, logic circuitry, and/or any devicethat manipulates 
digital information based on operational instruction, or 
in a predefined manner are examples of a data proces- 
sor. 

50 [0049] A data processor program in accordance with 
an embodiment of the disclosures made herein is 
processible by a data processor of an active and/or in- 
active routing module of a network element. A copy of 
the data processor program may be resident on each of 

55 the routing elements in a network element. Furthermore 
each copy of the data processor program may be ac- 
cessible by a data processor of the respective routing 
module from a memory apparatus of the respective rout- 
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ing module (e.g., RAM, ROM, virtual memory, hard drive 
memory, etc.) or from a peripheral apparatus such as a 
diskette, a compact disk, an external data storage de- 
vice and the like. 

[0050] A data processor program accessible from an 5 
apparatus by a data processor is defined herein as a 
data processor program product. It is contemplated 
herein that the data processor program product may 
comprise more than one data processor programs each 
accessible from respective apparatuses. It is further 10 
contemplated herein that each one of a plurality of data 
processor programs may be accessed by a different re- 
spective one of a plurality of data processors. For ex- 
ample, a first data processor and a second data proc- 
essor (e.g., of a leaf node and a root node), respectively, 15 
may access a first data processor program and a sec- 
ond data processor program, respectively, from a first 
apparatus and a second apparatus (e.g., a first memory 
device and a second memory device), respectively. 
[0051] In the preceding detailed description, refer- 20 
ence has been made to the accompanying drawings 
that form a part hereof, and in which are shown by way 
of illustration specific embodiments in which the inven- 
tion may be practiced. These embodiments have been 
described in sufficient detail to enable those skilled in 25 
the art to practice the invention, and it is to be under- 
stood that other embodiments may be utilized and that 
logical, mechanical, chemical and electrical changes 
may be made without departing from the spirit or scope 
of the invention. To avoid detail not necessary to enable 30 
those skilled in the art to practice the invention, the de- 
scription omits certain information known to those of skill 
in the art. The preceding detailed description is, there- 
fore, not to be taken in a limiting sense, and the scope 
of the present invention is defined only by the appended 35 
claims. 



Claims 

40 

1. A method for providing redundant protocol proc- 
esses in a network element, comprising: 

receiving at least one lower layer protocol pack- 
et at a lower layer task of a first module, wherein 45 
the at least one lower layer protocol packet in- 
cludes a higher layer protocol packet encapsu- 
lated therein; 

determining a peer network element from which 
the higher layer protocol packet was received so 
in response to failure of a higher layer protocol 
task of the first module while processing the 
higher layer protocol packet; 
terminating higher layer and lower layer proto- 
col sessions of a second module with the peer 55 
network element from which the higher layer 
protoco I packet was received in response to de- 
termining the peer network element from which 



the higher layer protocol packet was received; 
and 

implementing an activity switch from the first 
module to the second module after terminating 
the higher layer and the lower layer protocol 
sessions with the peer network element from 
which the higher layer protocol packet was re- 
ceived. 

2. The method of claim 1 wherein: 

the at least one lower layer protocol packet is 
a Transmission Control Protocol (TCP) packet; 
and 

the higher layer protocol packet is a Border 
Gateway Protocol (BGP) packet. 

3. The method of claim 1 wherein: 

the at least one lower layer protocol packet is 
a Transmission Control Protocol (TCP) packet; 
and 

the higher layer protocol packet is a Multi-Pro- 
tocol Label Switching (MPLS) packet. 

4. The method of claim 1 wherein determining the 
peer network element from which the higher layer 
protocol packet was received includes accessing a 
record of the peer network element maintained by 
the higher layer protocol task of the first module. 

5. The method of claim 4 further comprising: 

communicating with the second module the 
higher layer and lower layer protocol sessions 
to be terminated. 

6. The method of claim 1 wherein the higher layer 
and the lower layer protocol sessions are a Border 
Gateway Protocol (BGP) task session and a Trans- 
mission Control Protocol (TCP) task session, re- 
spectively. 

7. The method of claim 1 wherein terminating the 
lower layer protocol session includes removing a 
copy of the at least one lower layer protocol packet 
from a receive queue of the second module. 

8. The method of claim 1 , further comprising: 

synchronizing the first module with the second 
module after terminating the higher layer and 
the lower layer protocol sessions of the second 
module with the peer network element from 
which the higher layer protocol packet was re- 
ceived; and 

implementing higher layer and lower layer pro- 
tocol sessions between the second module and 
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the peer network element from which the higher 
layer protocol packet was received after syn- 
chronizing the first module with the second 
module. 

5 

9. The method of claim 8 wherein synchronizing the 
first module with the second module includes syn- 
chronizing at least a portion of information in a rout- 
ing information database of the first module with 
corresponding information in a routing information 10 
database of the second module. 

10. The method of claim 9 wherein the synchroniz- 
ing the first module with the second module further 
includes synchronizing at least a portion of config- is 
uration information and state information of the first 
module with corresponding configuration informa- 
tion and corresponding state information of the sec- 
ond module. 

20 

11. The method of claim 1 further comprising: 

receiving other lower layer protocol packets at 
the lower layer task of the first module, wherein 
the other lower layer protocol packets include 25 
other higher layer protocol packets encapsulat- 
ed therein; and 

processing the other higher layer protocol 
packets in the second module after processing 
the other higher layer protocol packets in the 30 
first module. 

12. The method of claim 11 wherein the step of 
processing the other higher layer protocol packets 
further comprises: 35 

processing the other higher layer protocol 
packets in the second module sufficiently after 
processing the other higher layer protocol 
packets in the first module so as to provide an *o 
ability to prevent the second module from 
processing at least one of the other higher layer 
protocol packets when the at least one of the 
other higher layer protocol packets results in 
failure of the higher layer protocol task of the 45 
first module. 

11. Apparatus for providing redundant protocol 
processes in a network element, comprising: 

50 

a first module for receiving at least one lower 
layer protocol packet at a lower layer task, 
wherein the at least one lower layer protocol 
packet includes a higher layer protocol packet 
encapsulated therein, and for determining a 55 
peer network element from which the higher 
layer protocol packet was received in response 
to failure of a higher layer protocol task of the 



first module while processing the higher layer 
protocol packet; 

a second module coupled to the first module; 
and 

a system controller coupled to the first module 
and the second module, the system controller 
for initiating termination of higher layer and low- 
er layer protocol sessions of the second module 
with the peer network element from which the 
higher layer protocol packet was received in re- 
sponse to the determining the peer network el- 
ement from which the higher layer protocol 
packet was received and for implementing an 
activity switch from the first module to the sec- 
ond module after the initiating termination of the 
higher layer and the lower layer protocol ses- 
sions with the peer network element from which 
the higher layer protocol packet was received. 

12. The apparatus of claim 11 wherein: 

the at least one lower layer protocol packet is 
a Transmission Control Protocol (TCP) packet; 
and 

the higher layer protocol packet is a Border 
Gateway Protocol (BGP) packet. 

13. The apparatus of claim 11 wherein: 

the at least one lower layer protocol packet is 
a Transmission Control Protocol (TCP) packet; 
and 

the higher layer protocol packet is a Multi-Pro- 
tocol Label Switching (MPLS) packet. 

14. The apparatus of claim 11 wherein the system 
controller accesses a record of the peer network el- 
ement maintained by the higher layer protocol task 
of the first module. 15.The apparatus of claim 14 
wherein the system controller communicates with 
the second module to determine the higher layer 
and lower layer protocol sessions to be terminated. 

16. The apparatus of claim 11 wherein the higher 
layer and the lower layer protocol sessions are a 
Border Gateway Protocol (BGP) task session and 
a Transmission Control Protocol (TCP) task ses- 
sion, respectively. 

17. The apparatus of claim 11 wherein the system 
controller causes a copy of the at least one lower 
layer protocol packet to be removed from a receive 
queue of the second module. 

1 8. The apparatus of claim 1 1 wherein the first mod- 
ule synchronizes with the second module after the 
termination of the higher layer and the lower layer 
protocol sessions of the first module with the peer 
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network element from which the higher layer proto- 
col packet was received, and wherein the second 
module implements higher layer and lower layer 
protocol sessions with the peer network element 
from which the higher layer protocol packet was re- 5 
ceived after synchronization of the first module with 
the second module. 

19. The apparatus of claim 1 8 wherein the synchro- 
nization of the first module with the second module 10 
includes synchronizing at least a portion of informa- 
tion in a routing information database of the first 
module with corresponding information in a routing 
information database of the second module. 

15 

20. The apparatus of claim 1 9 wherein the synchro- 
nization of the first module with the second module 
further includes synchronizing at least a portion of 
configuration information and state information of 

the first module with corresponding configuration in- 20 
formation and corresponding state information of 
the second module. 

21 . The apparatus of claim 1 1 wherein the first mod- 
ule further receives other lower layer protocol pack- 25 
ets at the lower layer task, wherein the other lower 
layer protocol packets include other higher layer 
protocol packets encapsulated therein, wherein the 
second module is adapted to process the other 
higher layer protocol packets after the first module 30 
processes the other higher layer protocol packets.. 

22. The apparatus of claim 21 wherein the second 
module is adapted to process the other higher layer 
protocol packets sufficiently after the first module 35 
processes the other higher layer protocol packets 

so as to provide an ability to prevent the second 
module from processing at least one of the other 
higher layer protocol packets when the at least one 
of the other higher layer protocol packets results in *o 
failure of the higher layer protocol task of the first 
module. 

23. The apparatus of claim 11 wherein the system 
controller is implemented within the first module, 45 
and a corresponding system controller is imple- 
mented within the second module, wherein func- 
tionality of the system controller is provided by the 
corresponding system controller upon the activity 
switch. 50 
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